Fix pytest CI matrix compatibility and prevent hangs#26027
Merged
Conversation
Copilot
AI
changed the title
[WIP] Fix CI configuration for pre-release pytest
Install matrix-specific pytest after test-requirements so pre-release job actually tests pre-release
Jul 2, 2026
…st-black compatibility
Use pytest 7.0.1 as the oldest supported lane on Python 3.10 and install matrix selections after shared requirements. Keep non-matrix builds on stable pytest, make manual plugin loading compatible across pytest versions, and fail bounded pipe harness waits instead of hanging CI indefinitely. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Cover Python 3.10 through 3.14 with the latest pytest 8.4 patch and the pytest pre-release channel. This keeps approximately one year of pytest compatibility coverage while detecting upcoming breaking changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
eleanorjboyd
force-pushed
the
copilot/fix-ci-pre-release-pytest
branch
from
July 21, 2026 17:37
279bf86 to
b9fe1a4
Compare
Keep the subprocess cwd annotation compatible with pathlib callers and simplify the local Black fixture for the supported pytest 8.4+ range so the CI Pyright version can analyze it cleanly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
pytest-black does not support pytest 8.1 or newer, and pytest-ruff already covers discovery of plugin-generated test items. Skip the obsolete Black-specific parameter instead of maintaining a local replacement for the third-party plugin. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR updates the Python test/CI setup so the GitHub Actions matrix actually runs the intended pytest versions (stable wildcard + pre-release), removes an obsolete pytest pin caused by pytest-black, and hardens the pytestadapter test harness to avoid indefinite hangs when subprocess/pipe communication fails.
Changes:
- Reorders/install-adjusts pytest in CI workflows and removes the global
pytest<8.1/pytest-blackconstraints that were downgrading matrix-selected pytest. - Adds bounded waits/timeouts and clearer failure propagation in
python_files/tests/pytestadapter/helpers.pyto prevent CI hangs. - Adds regression tests for subprocess timeout, pipe timeout, and failure surfacing; introduces a local replacement conftest for the black-related test data.
Show a summary per file
| File | Description |
|---|---|
| python_files/tests/pytestadapter/test_helpers.py | Adds regression tests covering subprocess timeout and pipe/failure handling. |
| python_files/tests/pytestadapter/test_coverage.py | Switches to PYTEST_DISABLE_PLUGIN_AUTOLOAD env var for cross-version compatibility. |
| python_files/tests/pytestadapter/helpers.py | Adds subprocess timeout and bounded pipe waits; refactors runner to avoid hang scenarios. |
| python_files/tests/pytestadapter/.data/2496-black-formatter/conftest.py | Provides a local, pytest-8.4+ compatible black-check collector for test data. |
| build/test-requirements.txt | Removes pytest<8.1 and pytest-black from shared test requirements. |
| .github/workflows/pr-check.yml | Expands Python versions and installs matrix-selected pytest after test requirements. |
| .github/workflows/build.yml | Ensures pytest is explicitly installed in the build workflow. |
Review details
- Files reviewed: 7/7 changed files
- Comments generated: 1
- Review effort level: Low
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
eleanorjboyd
marked this pull request as ready for review
July 21, 2026 20:21
The supported pytest 8.4+ range provides --disable-plugin-autoload, so use the option exercised by the regression test instead of the compatibility environment variable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
dmitrivMS
approved these changes
Jul 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix the Python test matrix so every job runs its requested pytest version, cover supported Python versions against recent and upcoming pytest releases, remove the obsolete global pytest pin, and prevent subprocess failures from hanging CI indefinitely.
Root cause
pytest-black==0.6.0uses the deprecatedpytest_collect_file(path, ...)hook removed by pytest 8.1. The resultingpytest<8.1workaround lived in shared test requirements, which were installed after the matrix-selected pytest version and silently downgraded stable and pre-release jobs to pytest 8.0.2.The downgrade also made the manual pytest-cov test reject
--disable-plugin-autoload. Its subprocess exited, but the named-pipe listener waited indefinitely, causing the multi-hour CI hangs.Test matrix
On Ubuntu and Windows:
pytest==8.4.*) for approximately one year of compatibility coverageThis produces 20 jobs. The 8.4 wildcard currently resolves to pytest 8.4.2.
Changes
pytest<8.1andpytest-blackfrom shared requirements.PYTEST_DISABLE_PLUGIN_AUTOLOADfor cross-version manual plugin loading.Validation
git diff --checkpass.